In the previous lesson, we decided on the architectural aspects of designing the Messenger API’s communication protocols and data formats. In this lesson, we will describe the endpoints, data entities, and the message format required for a chat between clients.

Base URL and API endpoints#

We will use the following URL for designing the Messenger API. The api.messenger.com is the host URL followed by the version v1.0 and the service.

The base URL for Messenger API
The base URL for Messenger API

The following illustration represents the operations and their associated endpoints:

GET /messages/{userID}?after_id={messageID}&limit={count} HTTP/2.0
POST /asset HTTP/2.0
GET /asset/{mediaFilesID} HTTP/2.0
GET /chat?encoding=text HTTP/2.0
Retrieve a list of messages
Retrieve or upload an asset
WebSocket connection
Endpoints for multiple operations in the Messenger API

Let's go over the purpose of the endpoints below:

  • WebSocket connection: This endpoint is used to establish a WebSocket connection with a chat server.

  • Retrieve or upload an asset: This endpoint is accessed to retrieve or upload a media file.

  • Retrieve a paginated list of messages: These endpoints are targeted to retrieve a paginated list of messages.

Remember: We don't have an endpoint for a live and offline chat because we use WebSockets for such communication. We will shortly see how that interaction between users takes place.

The message format for API endpoints#

In this section, we’ll discuss data entities, requests, and response structures for each functional requirement. We will also look at how a WebSocket connection is established to send one-to-one messages. Let's start our discussion with data entities.

Message data entities#

Following are the key data entities that are exchanged between the client and server:

Data entities relevant to a message that needs to be included in the message body

Establishing WebSocket connection#

Before sending a message, it is mandatory to have a WebSocket connection established with the chat server. Therefore, we must establish a connection with the chat server represented by the v1.0/chat endpoint. As shown below, the encoding parameter encoding=text shows that the messages will be exchanged in the UTF-8 encoding after the successful connection.

  • HTTP method: The WebSocket connection with the chat server is established by simply sending an HTTP GET request with some essential headers.

  • Request format: In the request body, we set the value of the Connection header to Upgrade and the value of the Upgrade header to websocket, indicating that we want to upgrade our current HTTP connection to the WebSocket protocol. The request is shown in the following code widget:

The HTTP request format to upgrade connection to WebSocket

The Sec-WebSocket-Version specifies a WebSocket version the client aims to use when communicating with the server. In the initial connection upgradation process, the client mentions the WebSocket versions that it can support via this header. For example, the client request could contain version 13 as shown below:

Sec-WebSocket-Version: 13

If the server supports version 13, the response from the server does not include the Sec-WebSocket-Version header. However, if the server doesn’t support the requested version, the response contains a 426 Upgrade Required error, and the Sec-WebSocket-Version header shows a list of values that a server supports. For example:

Sec-WebSocket-Version: 7, 8

The client then repeats a request to the server with the latest supported version amongst the list shared by the server, that is, version 8.

After the successful WebSocket connection establishment, the communication between the client and server starts using WebSocket version 8.

  • Response format: When the connection is successfully established, the server returns a response, as shown below. In the response, we see the status code 101, indicating that the WebSocket protocol has been successfully upgraded.

A successful response format to upgrade request

The following figure illustrates the process of WebSocket connection establishment:

The WebSocket connection establishment
The WebSocket connection establishment

In the above figure, we establish an HTTP connection with the chat server, which is then upgraded to a WebSocket connection.

Note: As soon as a WebSocket connection is established between a client and chat server, the latter communicates about the connection to the WebSocket manager, which updates its mapping table.

Live communication#

After establishing a WebSocket connection, let's see how text messages are sent over this connection in the JSON format. Before sending the data over the connection, the WebSocket API converts it to the UTF-8 encoded text format. Let's start with a message without an attachment.

Message without an attachment#

When the clients establish the WebSocket connection with the server, they can send and receive text messages to each other. A sample text message, along with its metadata, is shown below:

A sample message in JSON format that needs to be transmitted

When the server receives the message, it will send an acknowledgment back to the sender showing that it has received it. Line 9 in the following code widget shows that the attribute receivedServer is set to true in the response from the server. Similarly, the server responds with the delivered and read attributes as true when the receiver receives and reads the message.

A response from the server upon receiving and delivering a message

Points to Ponder

Question 2

What is the format of a message if we send multiple attachments in a chat?

Hide Answer

If a message has multiple attachments, we can represent them via a JSON list—for example, attachmentURI: ["URI-01", "URI-02", ...] as shown below:

{
"connection_id": "rLCkwH/SKGAsO9H/ZShrBAFTDKU=",
"payload": [
 {
  "senderId"       : "Rtnukkald=",         
  "recieverId"     : "IJUkalxUl=",                   
  "messageId"      : "Pqwnkilsx=",        
  "timestamp"      : "9:47PM 12-1-2023",     
  "attachement"    : true,
  "attachementURI" : ["/v1.0/asset/{id-01}", "/v1.0/asset/{id-02}", "..." ],
  "messageText"    : "Hello, Welcome!"         
   }
  ]
}

2 of 2

Retrieving offline messages#

When the presence server determines that some user's status is offline, the messages intended for them are stored in the database and marked undelivered. When a user comes online, the presence server receives a heartbeat event from them, the chat server is notified, and the WebSocket connection is established. In the next step, the messages are delivered via the WebSocket connection in the format shown below:

Note: he WebSocket protocol uses some control signals, such as ping and pong, that work as the heartbeat to check that the client is still responsive.

The response containing the offline messages to be delivered when a user becomes online

Point to Ponder

Question

How does searching a list of messages work?

Hide Answer

There are two scenarios to search a list of messages for a query:

  1. Client-side search: In this approach, the user’s messages are searched for the query locally on the client side. While the most recent messages are cached, older messages may be stored and later retrieved from a local database.

  2. Server-side search: In this approach, the search API is utilized using GET, where the user’s query is sent to the backend servers. At the backend, the user’s messages are searched for the query, and a relevant paginated list of messages is retrieved.

Note: The server-side search approach is generally used in tandem with the client-side storage. This is because the client shouldn’t be burdened by storing all the previous messages. However, clients often revisit recent messages, and therefore, for a good user experience, it makes sense to keep recent messages in local storage.

Retrieving a paginated list of messages on alternative devices#

While messages may be retrieved locally from a database, they are not readily available on alternative devices. In that case, the client may GET messages in the form of a paginated list. Let's discuss the request and response size for retrieving a paginated list of messages on other devices.

Note: We have a detailed lesson on the different types of pagination. You can revisit it here.

  • HTTP method: To retrieve a paginated list of messages, we use the GET method.

  • Request format: In the query string, we mentioned the userID representing the sender's window, from whom the messages are retrieved. The query string contains the after_id and limit, which shows the message ID and the number of messages we want to receive, respectively.

The HTTP request format to retrieve a paginated list of messages
  • Response format: The response from the server contains a list of messages in the body, as shown below:

The response format to retrieve a paginated list of messages

To retrieve a paginated list of messages, the body in each response consists of messages along with the message metadata, such as senderId, receiverId, messageId, timestamp, and so on.

Remember: We will use this endpoint only to retrieve a paginated list of messages. To perform real-time chatting via the alternative device, a WebSocket connection will still be required to establish two-way communication with the server.

Failed requests#

The following table indicates different status codes in case of errors while performing various operations relevant to sending or receiving messages:

Error Responses

Status Code

Phrase

Description

400

Bad Request

  • When a request does not comply to standard request format

401

Unauthorized

  • Represents a missing or invalid authentication token

403

Forbidden

  • When the user is not allowed to perform the operation or when the resource (chat or message) is unavailable

404

Not Found

  • Indicates that the desired operation couldn't be performed because the message doesn't exist


405


Method Not Allowed

  • Indicates that the request URL exists, but the HTTP method in the request is not applicable (for example, the request will give this error if our API doesn't allow the PUT method)

413

Payload Too Large

  • When the number of characters for a message exceeds the maximum allowed characters

426

Upgrade Required

  • Shows that to fulfill the request, the client must upgrade to either a different protocol or another version of the current protocol

500

Internal Server Error

  • A general error when a backend server throws an exception

Summary#

A summary of the request detail for each endpoint is shown in the table below. The two-way chat is performed via WebSocket protocol, and the rest of the operations are performed via HTTP methods.

Operations

Endpoints

Data Entities

Response

(Status code, body)

WebSocket connection

GET /v1.0/chat?encoding=text

No data entities

101 Switching Protocols , No

Retrieve an asset

GET /asset/{mediaFilesID}


mediaFilesID is passed as in the URL

200 OK, Yes


Upload an asset

POST /asset


The media file, and the relevant metadata is passed in message body

200 OK, Yes


Retrieve a paginated list of messages

GET /v1.0/messages/{userID}?after_id={messageId}&limit={count}

userID, messageId, and count parameters are passed in the URL

200 OK, Yes

Messenger API Design Decisions

Messenger API Design Evaluation and Latency Budget